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Atlas 5 prepped to launch NASA’s twin Radiation Belt Storm Probes 
or^30 August 2012 

(http://www.spaceflightnow.com/atlas/av032/status.html) 


This Quarter’s Highlights 

The AMU team worked on six tasks for their customers: 


Launch Support 

Dr. Watson and Dr. Huddleston 
supported the first Atlas 5 launch 
attempt on 24 August. 

Ms. Crawford and Dr. Huddle- 
ston supported the second Atlas 
V launch attempt on 25 August. 

Ms. Shafer and Dr. Huddleston 
supported the successful launch 
of the Atlas V on 30 August. 



Dr. Bauman delivered the final report describing his work on the objective lightning forecast task for 
the Kennedy Space Center/Cape Canaveral Air Force Station area. 

Ms. Crawford continued working on the objective lightning forecast task for airports in east-central 
Florida. 


Ms. Shafer began work on a new task for Vandenberg Air Force Base to create an objective and 
automated tool that will help forecasters relate pressure gradients to peak wind values. 

Dr. Bauman created a graphical user interface for the NASA Launch Services Program and 45th 
Weather Squadron to assess model forecasts of upper-level winds. 

Dr. Huddleston completed research to determine whether Global Position System precipitable wa- 
ter data could improve the lightning forecast. 


Dr. Watson returned from maternity leave and resumed testing high-resolution model configurations 
for Wallops Flight Facility to provide forecasters with more accurate depictions of the future state of 
the atmosphere. 
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This section contains summaries of the AMU activities for the fourth quarter of Fiscal Year 2012 (July- 
September 2012). The accomplishments on each task are described in more detail in the body of the report starting 
on the page number next to the task name. 


Objective Lightning Probability Forecast, Phase IV ( Page 5) 


Purpose : Develop updated equations with six more years of data 
and use the National Lightning Detection Network (NLDN) daily light- 
ning flash count across central Florida to determine if the data can be 
stratified by lightning sub-season instead of calendar month. If the 
data cannot be stratified by lightning sub-season, the monthly equa- 
tions will be updated with the new data. The 45th Weather Squadron 
(45 WS) uses the AMU-developed Objective Lightning Probability 
tool as one input to their daily lightning forecasts. Updating the lo- 
gistic regression equations with additional data and different stratifi- 
cations could improve the lightning probability forecast and make the 
tool more useful to operations. 

Accomplished: Completed and delivered the final report to the AMU 
customers and posted the report on the AMU website. 
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Objective Lightning Probability Forecasts for East-Central Florida 
Airports ( Page 5) 
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Purpose : Develop an objective lightning probability 
forecast tool for commercial airports in east-central 
Florida to help improve the lightning forecasts during 
the day in the warm season. The forecasters at the Na- 
tional Weather Service in Melbourne, Fla. (NWS MLB) 
are responsible for issuing forecasts for airfields in cen- 
tral Florida, and need to make more accurate lightning 
forecasts to help alleviate delays due to thunderstorms 
in the vicinity of an airport. The AMU will develop a 
forecast tool similar to that developed for the 45 WS in 
previous AMU tasks. The probabilities will be valid for 
the areas around the airports and time periods needed 
for the NWS MLB forecast. 

Accomplished: Created and tested July lightning 
probability forecast equations for Orlando (MCO) and 
Melbourne (MLB) International Airports and deter- 
mined their performance was not adequate. Met with 
NWS MLB forecasters to determine next steps and 
began developing the graphical user interface (GUI). 


AMU Quarterly Report 


2 


July — September 2012 



Quarterly Task Summaries 

(continued) 


Assessing Upper-level Winds on Day-of-Launch (Page 7 ) 


Purpose: Develop a Meteorological Interactive Data Display System 
(MIDDS) or Excel capability to rapidly assess the model forecast of upper- 
level winds by calculating the differences between model data and the cur- 
rent upper-level wind speed and direction observations from the 50 MHz 
Doppler Radar Wind Profiler and Automated Meteorological Profiling Sys- 
tem (AMPS). This capability will provide an objective method for the launch 
weather officers (LWO) to compare the forecast upper-level winds to the 
observed data and assess the model potential to accurately forecast 
changes in the upper-level profile through the count. 

Accomplished: Completed Excel GUI based on 45 WS LWOs feedback. 
Addressed two Information Technology challenges: software compatibility 
and data access. The 45 WS installed the required software on one of their 
computers, and several solutions were proposed that will allow 45 WS ac- 
cess to the model data needed for the tool. Completed the final report. 



Vandenberg AFB Pressure Gradient Wind Study (Page 9 ) 


Purpose: Provide a wind forecasting capability that will improve 
wind warning forecasts and enhance the safety of the 30th Oper- 
ational Support Squadron (30 OSS) customers’ operations. This 
capability will be an Excel GUI that ingests surface pressure data 
automatically and determine the likelihood of reaching warning- 
level winds based on the pressure gradient across Vandenberg 
Air Force Base (VAFB). This will allow 30 OSS forecasters to 
evaluate pressure gradient thresholds between specific pairs of 
regional observing stations under different synoptic regimes to 
help determine the onset and duration of warning category 
winds. 

Accomplished: Began working with the 30 OSS to acquire ob- 
servations from 26 VAFB wind towers. Started writing Perl scripts 
to develop a database containing the maximum hourly peak 
winds for each day from 2007-2012. 
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Quarterly Task Summaries 

(continued) 

Using Global Positioning System Integrated Precipitable Water Vapor to 
Forecast Lightning on KSC/CCAFS ( Page 10 ) 

Purpose: Use output from the 45 WS Objective Lightning 
Probability tool, the current Global Positioning System 
(GPS) integrated precipitable water vapor (IPW) value, 
and the change in the GPS-IPW value over the last 0.5 to 
24 hours (in 30-minute increments) to determine the time 
period for the GPS-IPW change that produces the best 
probability forecast. The output from the combined Objec- 
tive Lightning Probability/I PW tool will be compared to the 
output of the Objective Lightning Probability tool alone to 
determine the value added, if any, to lightning prediction 
capability. If the value added is sufficient, the AMU will 
develop a forecast tool using the Objective Lightning 
Probability tool output and the IPW data as input. 

Accomplished: Completed the data analysis and found 
that GPS-IPW provided no added value to the lightning 
probability forecasts. Drafted the final report. 



Range-Specific High-Resolution Mesoscale Model Setup ( Page 14) 


Purpose: Establish a high-resolution model for the Eastern 
Range (ER) and Wallops Flight Facility (WFF) to better fore- 
cast a variety of unique weather phenomena. Global and 
national scale models cannot properly resolve important lo- 
cal-scale weather features due to their coarse horizontal res- 
olutions. A properly tuned model at a high resolution would 
provide that capability and provide forecasters with more 
accurate depictions of the future state of the atmosphere. 

Accomplished: Completed all model test cases for WFF 
using several Weather Research and Forecasting (WRF) 
model domain configurations, and validated and compared 
all output against local observations. The results were 
mixed, the configurations performed similarly. 
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AMU ACCOMPLISHMENTS DURING THE PAST QUARTER 

The progress being made in each task is provided in this section, organized by topic, 
with the primary AMU point of contact given at the end of the task discussion. 


SHORT-TERM FORECAST IMPROVEMENT 


Objective Lightning 
Probability Forecast - 
Phase IV (Dr. Bauman 
and Ms. Crawford) 

The 45 WS includes the probabil- 
ity of lightning occurrence in their 
daily morning briefings. This forecast 
is important in the warm season 
months, May-October, when the area 
is most affected by lightning. The 
forecasters use this information 
when evaluating launch commit crite- 
ria and planning for daily ground op- 
erations on Kennedy Space Center 
(KSC) and Cape Canaveral Air Force 
Station (CCAFS). The daily lightning 
probability forecast is based on the 
output from an objective lightning 
forecast tool developed in two phas- 
es by the AMU that the forecasters 
supplement with subjective analyses 
of model and observational data. The 
tool developed in Phase II consists of 
a set of equations, one for each 
warm season month, that calculates 


the probability of lightning occur- 
rence for the day more accurately 
than previous forecast methods 
(Lambert and Wheeler 2005, Lam- 
bert 2007). The equations are ac- 
cessed through a graphical user in- 
terface in the 45 WS primary weather 
analysis and display system, MIDDS. 
The goal of Phase III was to create 
equations based on the progression 
of the lightning season as seen in the 
daily climatology instead of an equa- 
tion for each month in order to cap- 
ture the physical attributes that con- 
tribute to thunderstorm formation. 
Five sub-seasons were discerned 
from the daily climatology, and the 
AMU created and tested an equation 
for each. The Phase III equations did 
not outperform Phase II. Therefore, 
the Phase II equations are still in op- 
erational use. For this phase, the 45 
WS requested the AMU make anoth- 
er attempt to stratify the data by light- 
ning sub-season. The AMU did this 
by using lightning observations 
across central Florida from NLDN. 


After an extensive analysis, Dr. Bau- 
man determined the NLDN-based 
lightning sub-seasons were unidenti- 
fiable, so he created monthly equa- 
tions with six more years of data than 
used in Phase II. The new equations 
did not outperform those from Phase 
II and will not be transitioned to oper- 
ations with the exception of the Octo- 
ber equation that does not currently 
exist in the Phase II operational tool. 

Final Report 

Dr. Bauman completed the final 
report. After the KSC Export Control 
Office completed the Scientific and 
Technology Information assessment, 
he distributed the report to the AMU 
customers and Ms. Crawford posted 
the final report on the AMU website, 
at http://science.ksc.nasa.gov/amu/ . 

For more information contact Dr. 
Bauman at bauman.bill@ensco.com 
or 321-853-8202, or Ms. Crawford at 

crawford.winnie@ensco.com or 321- 
853-8130. 


Objective Lightning 
Probability Forecasts 
for East-Central Florida 
Airports (Ms. Crawford 
and Dr. Bauman) 

The forecasters at NWS MLB are 
responsible for issuing weather fore- 
casts to several airfields in central 
Florida. They identified a need to 
make more accurate lightning fore- 
casts to help alleviate delays due to 
thunderstorms in the vicinity of an 
airport. Such forecasts would also 
provide safer ground operations 


around terminals, and would be of 
value to Center Weather Service 
Units serving air traffic controllers in 
Florida. To improve the forecast, the 
AMU was tasked to develop an ob- 
jective lightning probability forecast 
tool for the commercial airports in 
east-central Florida for which NWS 
MLB has forecast responsibility using 
data from the NLDN. The resulting 
forecast tool will be similar to that 
developed by the AMU for the 45 WS 
in previous tasks (Lambert and 
Wheeler 2005, Lambert 2007). The 
lightning probability forecasts will be 
valid for the time periods and area 
around each airport needed for the 


NWS MLB forecasts in the wet sea- 
son months, defined as May- 
September. 

MCO and MLB July Equations 

After learning that the MCO May 
and June equations developed from 
the new period of record (POR) with 
the new flow regime candidate pre- 
dictors did not show improved perfor- 
mance over the previous equations 
(AMU Quarterly Report Q3 FY12), 

Mr. Volkmer of NWS MLB asked Ms. 
Crawford to develop and test July 
equations for MCO and MLB. Both 
Mr. Volkmer and Ms. Crawford 
thought the equations for July might 
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perform better because this month 
had the highest percentage of light- 
ning occurrence in the POR. They 
expected that more lightning days in 
the data set was more likely to result 
in robust relationships between the 
predictors and lightning occurrence. 
The results would help them deter- 
mine whether to continue with MCO 
equation development. 

Ms. Crawford calculated the per- 
cent improvement or degradation in 
skill of the new equations over five 
forecast benchmarks using the Brier 
Skill Score (SS) defined in Wilks 
(2006). Table 1 contains the SS val- 
ues showing the skill of the July 
MCO and MLB equations using the 
development and verification da- 
tasets. In general, equations perform 
better when using the data from 
which they were developed and not 
as well using an independent verifi- 
cation dataset. Ms. Crawford used 
these two datasets as 
input to determine if this 
was the case for the July 
equations. 

The positive values 
in Table 1 indicate the 
equations had more skill 
than the corresponding 
forecast method, and 
negative values indicate 
less skill. As in all previ- 
ous tests, the equations 
outperformed 1-day per- 
sistence. However, the 
results for the daily cli- 
matology and flow re- 
gime probabilities were 
mixed when using either 
the development or veri- 
fication dataset. Values 
with magnitudes within 
10% of 0, positive or 
negative, likely indicate 
that the equations per- 
formed similarly to the 
corresponding forecast 
method. The general rule 
of development data per- 
forming better than verifi- 
cation data held in this 
case. The development 
data SS values were 
higher and all positive, 

V 


but a large percentage of the values 
were < 10%. A majority of the verifi- 
cation SS values were within 10%, 
and several of those were negative. 
That both datasets produced results 
showing similar performance be- 
tween the forecast benchmarks and 
the equations indicated that the 
equations did not provide added val- 
ue to the forecast. 

Area and Time Stratifications 

To help determine why the MLB 
equations did not perform well, Ms. 
Crawford compared the area and 
time stratifications to the 45 WS’s 
Objective Lightning Probability tool 
developed by the AMU (Lambert 
2007). The area in the 45 WS tool is 
larger than the 10 NM radius around 
MLB and MCO, and the time period 
was 17 hours (0700 to midnight local 
time) as opposed to the four three- 
hour time periods during 1500 to 


\ 

0300 UTC, which totals 12 hours 
(1100 to 2300 local time). If lightning 
occurred just outside of the 10 NM 
radius or just before or after a time 
period, a robust relationship between 
predictors and the predictand might 
not be fully realized by the equations. 

Ms. Crawford and Dr. Huddleston 
spoke with with Mr. Sharp and Mr. 
Volkmer of NWS MLB and suggest- 
ed increasing the area and/or time 
period to help improve equation per- 
formance. Dr. Huddleston found sim- 
ilar issues with the time stratification 
in another AMU task to use GPS 
IPW estimates to forecast lightning 
(AMU Quarterly Report Q3-FY12). 

Mr. Sharp and Mr. Volkmer said the 
area cannot be changed, but they 
would consider lengthening the time 
periods. In a later meeting with Ms. 
Crawford and Dr. Bauman, they re- 
quested that the equations be devel- 
oped for MCO and MLB using the 


Table 1. The percent improvement (positive black font) or degradation (negative red 
font) in skill of the MCO and MLB July equations over the forecast benchmarks of 1- 
day persistence, daily climatology and three flow regime probabilities. Scores were 
calculated using the development and verification data set for each station. Cells 


shaded in yellow contain values within 10% of 0, not inclusive. 


Station 

Dataset 

Forecast Benchmark 

15-18 

18-21 

21-00 

00-03 

MCO 

Development 

1-Day Persistence 

52 

44 

48 

56 

Daily Climatology 

14 

13 

13 

6 

Flow Regime Probability 

11 

11 

6 

3 

Flow Regime 2-Speed 

8 

11 

6 

4 

Flow Regime 3-Speed 

9 

10 

4 

3 

Verification 

1-Day Persistence 

48 

51 

50 

48 

Daily Climatology 

-6 

8 

28 

4 

Flow Regime Probability 

-7 

5 

23 

-3 

Flow Regime 2-Speed 

0 

5 

20 

-5 

Flow Regime 3-Speed 


4 

12 


MLB 

Development 

1-Day Persistence 

48 

47 

49 

49 

Daily Climatology 

15 

21 

22 

10 

Flow Regime Probability 

13 

9 

6 

7 

Flow Regime 2-Speed 

12 

9 

6 

4 

Flow Regime 3-Speed 

11 

9 

4 

5 

Verification 

1-Day Persistence 

53 

45 

54 

51 

Daily Climatology 

-4 

-7 

10 

1 

Flow Regime Probability 

-8 

-9 

3 

-3 

Flow Regime 2-Speed 

-1 

-4 

-7 

2 

Flow Regime 3-Speed 


-9 

-2 

-4 
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^original requested area and time period stratifications, 
even if they did not perform significantly better than the 
daily and flow regime climatologies. 

Excel GUI 

Mr. Sharp and Mr. Volkmer requested that the GUI to 
be created with the task include the daily and flow re- 
gime climatologies as well as the equation output for 
each month. Ms. Crawford began creating a GUI to dis- 
play the daily climatology and flow regime probability 
values and will add input and output for the equations 
when they are finished. Figure 1 shows the initial form in 
the GUI to input values needed to output all the climatol- 
ogy values. The month, day, station and time are need- 
ed for the daily climatology, these values plus the flow 
regime are needed for the flow regime probabilities, and 
all values including the speed are needed for the speed- 
stratified flow regime probabilities. 

Status 

Ms. Crawford finished a rough draft of the GUI to 
output values for the daily and flow regime climatologies. 
She is preparing and testing it before showing it to Mr. 
Volkmer for his input on the design and functionality. 

For more information contact Ms. Crawford at 321- 
853-8130 or crawford. winnie@ensco.com . or Dr. Bau- 
man at 321-853-8202 or bauman.bill @ensco.com . 



Figure 1. The initial form in the GUI to input the values 
needed to output the climatology values. 


Assessing Upper-Level 
Winds on Day-of- 
Launch (Dr. Bauman) 

On the day-of-launch, the 45 WS 
LWOs monitor the upper-level winds 
for their launch customers to include 
NASA’s Launch Services Program. 
They currently do not have the capa- 
bility to display and overlay profiles of 
upper-level observations and numeri- 
cal weather prediction model fore- 
casts. The LWOs requested the AMU 
to develop a capability in the form of 
a GUI that will allow them to plot up- 
per-level wind speed and direction 
observations from the KSC 50 MHz 
wind profiler and CCAFS AMPS radi- 
osondes, and then overlay forecast 
profiles from the North American 
Mesoscale (NAM), Rapid Update Cy- 
cle (RUC), which is now the Rapid 
Refresh (RAP) and Global Forecast 
System (GFS) models to assess the 
performance of these models. 

Excel GUI 

Dr. Bauman completed the final 
version of the GUI after demonstrat- 
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ing it to the 45 WS LWOs. The final 
version shown in Figure 2 consists of 
three major changes based on the 
LWOs feedback. First, the back- 
ground color of the charts was 
changed from white to gray so the 
wind speed and wind direction lines 
will stand out against the back- 
ground. Second, the 50 MHz profiler 
and rawinsonde observations are 
displayed as solid dark red lines and 
the 915 MHz profiler as solid orange 
lines; and the forecasts are displayed 
as dashed blue lines trending from 
light blue for the first forecast valid 
time to dark blue for the last forecast 
valid time. Finally, the number and 
interval of each model’s forecast dis- 
played are limited to four at three- 
hour intervals for clarity. 

Information Technology 

Information Technology (IT) chal- 
lenges in this task included incompat- 
ible versions of commercial off-the- 
shelf software and changing IT secu- 
rity requirements. These issues im- 
pacted the ability of the software to 
produce the required results and 
changed data acquisition methods. 


Software Compatibility 

Dr. Bauman used Excel 2010 to 
develop the GUI because it was part 
of the standard AMU software load 
as part of Microsoft Office 2010 as 
approved by NASA IT System Ad- 
ministration and Security. After Dr. 
Bauman first demonstrated the GUI 
to the LWOs, they suggested that the 
he conduct tests of the tool on the 
45th Space Wing (45 SW) network. 
Upon doing so, Dr. Bauman realized 
that most of the charting functionality 
used in the Excel 2010 version of the 
tool was not backwards compatible 
with the Excel 2007 version on the 45 
SW network. Only Excel 2010 pos- 
sessed the capabilities to automati- 
cally format the charts to meet the 
requirements set forth in this task. To 
solve this issue, Dr. Bauman request- 
ed the 45 WS install Excel 2010, 
which they did after getting permis- 
sion from 45 SW IT Security. 

Data Access 

Testing the tool on the 45 SW 
network also revealed that 45 SW IT 
Security did not permit 45 SW com- 
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Figure 2. Wind speed (left) and wind direction (right) profiles from the 1655 UTC 13 August 2012 50 
MHz profiler observation (dark red lines), 1701 UTC 13 August 2012 915 MHz profiler observation 
(orange lines) and NAM model forecasts (dashed blue lines) valid every three hours from 1900 UTC 13 
August 2012 through 0400 UTC 14 August 2012 plotted in Excel. The legend for data type, date and 
time are shown in the upper left of the chart. The legend text colors match the line colors in the charts. 


puters to access the Iowa State Uni- 
versity Weather Archive server 
where the model forecast data files 
were located. The 45 WS suspected 
this would be an issue as access 
to .edu Universal Resource Locators 
(URL) is normally blocked by IT Se- 
curity. Since Dr. Bauman’s testing 
also revealed that access to the 
Spaceport Weather Data Archive, 
a .gov URL, was permitted, they 
asked the KSC Weather Office if the 
Spaceport Weather Data Archive cu- 
rator, Mr. Gemmer of Abacus Tech- 
nology, could routinely retrieve the 
model files from the Iowa State Uni- 
versity Weather Archive server and 
save them in a directory on the 
Spaceport Weather Data Archive for 
access by the 45 WS. Mr. Gemmer 
notified the KSC Weather Office that 
they could retrieve and save the 
model files for 45 WS access. 


When this task was about 75% 
complete, the KSC Weather Office 
notified Dr. Bauman that the Space- 
port Weather Data Archive was be- 
ing phased out and transitioning to 
the Spaceport Weather Archive at a 
new URL: 

http://wxarchive.ksc.nasa.gov 

The AMU tested access to the new 
site via the File Transfer Protocol 
(FTP) being used in the Excel Visual 
basic for Applications (VBA) scripts 
and found that the directory structure 
used on the old site was not the 
same on the new site. Upon manual 
inspection of the new URL, Dr. Bau- 
man discovered that the new site is 
built only for human interface to 
search and acquire data, not for au- 
tomated retrieval of data by another 
computer. Dr. Bauman asked Mr. 
Gemmer for assistance and he indi- 
cated that due to NASA IT security 


regulations the 
FTP portion of 
the server would 
be shut down, but 
they could setup 
a directory struc- 
ture to allow the 
automated trans- 
fer of files via Hy- 
pertext Transfer 
Protocol (HTTP). 
This was not an 
issue because 
Dr. Bauman 
would only have 
to modify the 
VBA code to use 
HTTP instead of 
FTP for the cor- 
rect directory 
structure on the 
server. 

Based on resolu- 
tion of the chal- 
lenges presented 
in the preceding 
paragraphs, Dr. 
Bauman contin- 
ued to develop 
the GUI and com- 
plete the task. 
After final testing 
on the NASA net- 
work and prior to transferring the tool 
to the 45 WS computer for testing, 

Dr. Bauman checked on the status of 
the HTTP directory structure on the 
new Spaceport Weather Archive 
URL so they could update the VBA 
code. Mr. Gemmer informed him that 
since his last communication, NASA 
IT Security was shutting down HTTP 
servers thereby disallowing the tool 
to automatically access any files on 
the Spaceport Weather Archive. Dr. 
Bauman and Dr. Huddleston met 
with Mr. Ebuen and Ms. Burdett of 
Abacus Technology to determine if 
there was another solution to allow 
the tool to automatically access the 
data on the servers. Four possible 
solutions were discussed: 

• Use HTTP Secure access with 
password protection, 

• Run the Excel GUI on the Space- 
port Weather Archive server, 
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• Implement a Web service, which 
is a software system designed to 
support interoperable machine-to 
-machine interaction over a net- 
work, and 

• Implement a Server Message 
Block or Common Internet File 
System that operates as an appli- 
cation-layer network protocol to 
provide shared access to files 
between nodes on a network. 


After assessing the four options, 
Mr. Gemmer, Mr. Ebuen and Ms. 
Burdett thought implementing a Web 
service would be the best option. 
Based on this assessment and work- 
ing with Mr. Gemmer, Dr. Huddleston 
submitted an IT Work Request to 
NASA/KSC IT Security to request 
support to provide a solution. Once a 
solution is in-place, Dr. Bauman will 
work with the 45 WS system admin- 
istrators to install the GUI on a 45 
WS computer with Excel 2010 and 
test it on the 45 SW network. 


Final Report 

Dr. Bauman completed the final 
report. After the KSC Export Control 
Office completes the Scientific and 
Technology Information assessment, 
the AMU will post the final report on 
the AMU website. 

For more information contact Dr. 
Bauman at bauman.bill@ensco.com 

or 321-853-8202. 



Vandenberg AFB 
Pressure Gradient Wind 
Study (Ms. Shafer) 

Warning category winds can ad- 
versely impact day-to-day space lift 
operations at VAFB. For example, 
winds > 30 kt can affect Delta II vehi- 
cle transport to the launch pad, Delta 
IV stage II attitude control system 
tank load, and other critical opera- 
tions. The 30th Operational Support 
Squadron (30 OSS) forecasters at 
VAFB use the mean sea level pres- 
sure from seven regional observing 
stations to determine the magnitude 
of the pressure gradient as a guide to 
forecast surface wind speed at 
VAFB. Their current method uses an 
Excel-based tool that is manually in- 
tensive and does not contain an ob- 
jective relationship between peak 
wind and pressure gradient. They 
require a more objective and auto- 
mated capability to help them fore- 
cast the onset and duration of warn- 
ing category winds to enhance the 
safety of their customers’ operations. 
The 30 OSS has requested that the 
AMU develop an automated Excel 
GUI that includes pressure gradient 
thresholds between specific observ- 
ing stations under different synoptic 
regimes to aid forecasters when issu- 
ing wind warnings. 

Current 30 OSS Tool 

Mr. Brock of 30 OSS provided 
Ms. Shafer with an Excel file contain- 
ing the current 30 OSS tool used by 
the forecasters to determine pressure 
^gradients between specific regional 


observing stations. She ex- 
amined the contents of the 
file to better understand their 
needs and identified the 7 
observing stations (Figure 3) 
and the 12 station pairs they 
use to determine the pres- 
sure gradient. The 12 pairs 
taken from Figure 3 are 
KVBG - KBFL 
KBFL - KLAS 
KVBG - KLAS 
KACV - KSFO 
KSFO - KPRB 
KPRB - KVBG 
KVBG - KLAX 
KACV - KPRB 
KPRB - KLAX 
KACV - KVBG 
KSFO - KVBG 
KACV - KLAX 

Ms. Shafer and Dr. Bauman not- 
ed the current tool requires several 
manual inputs from the user through- 
out the day and discussed the ability 
to automate it. Mr. Brock agreed this 
would be a great advantage for the 
forecasters and would like to see this 
capability in the new pressure gradi- 
ent tool. Ms. Shafer will use the cur- 
rent 30 OSS tool as a starting point 
and will upgrade it to include automa- 
tion and the results of the pressure 
gradient assessment. 

Data Acquisition and Processing 

Part of this task is to determine 
past high wind events and compare 
the local pressure gradients at the 
time. Ms. Shafer will collect the his- 


torical wind data from the 26 VAFB 
wind tower network and observations 
from each of the regional observing 
stations will be collected from the Na- 
tional Climatic Data Center archive. 

In order to confirm the wind tower 
data is in a format the AMU could 
easily process, Mr. Brock provided 
Ms. Shafer sample wind data files 
collected from two of the VAFB wind 
towers. She began writing Perl 
scripts to determine the hourly 4 me- 
ter peak wind for each tower on a 
given day, which will be used to de- 
velop the peak wind database. 

Contact Ms. Shafer at 321-853- 
8200 or shafer.iaclvn@ensco.com for 

more information. 
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INSTRUMENTATION AND MEASUREMENT 


Using GPS-IPW to 
Forecast Lightning on 
KSC/CCAFS 
(Dr. Huddleston) 

The 45 WS forecasters include a 
probability of lightning occurrence in 
their daily 24-hour and weekly plan- 
ning forecasts. This value is used by 
personnel involved in determining the 
possibility of violating launch commit 
criteria and planning for daily ground 
operation activities on KSC/CCAFS. 
To help improve this forecast, the 
AMU developed the 45 WS’s Objec- 
tive Lightning Probability tool, which 
is used every day during the warm 
season to forecast the probability of 
lightning occurrence for the day. This 
tool outperformed the 45 WS’s previ- 
ous objective lightning probability 
technique by 56% (Lambert 2007). 
The 45 WS and others have also in- 
vestigated techniques using GPS- 
IPW observations and changes over 
specified time periods to predict the 
probability of lightning, each showing 
promising results (Mazany et al. 

2002; Inoue and Inoue 2007; Kehrer 
et al. 2008; Suparta et al. 2011a; and 
Suparta et al. 2011b). In this task, the 
AMU determined the utility of using 
GPS-IPW and output from the Objec- 
tive Lightning Probability tool to pre- 
dict the probability of lightning at the 
temporal resolution of the GPS-IPW, 
which is every 30 minutes. 

Logistic Regression 

After an exploratory data analy- 
sis, Dr. Huddleston determined that a 
multiple, logistic regression model 
was the best choice for this study. 
Logistic regression is the appropriate 
model to use when the predictand, or 
element to be forecast, is binary. In 
this case, the element to be forecast 
was lightning occurrence: yes, or 1, 
meant lightning occurred and no, or 
0, meant lightning did not occur with- 
in the specified time period and area 
of interest. As in Kehrer et al. (2008), 
Dr. Huddleston developed 2-hour 
and 9-hour forecast models. These 

V 


models were limited 
to the hours be- 
tween 0700-0000 
EDT to be con- 
sistent with the Ob- 
jective Lightning 
Probability tool. The 
output of the logistic 
regression equation 
is a lightning index 
that gives the proba- 
bility that lightning 
will occur. 

Predictor Selection 
Methodologies 

Because there 
were 50 candidate 
predictors for these 
models, the goal 
was to determine a 
subset of predictors 
that affected the 
predictand to create 
a model that fits the 
data well without the 
negative effects of 
overfitting the mod- 
el. Models that are 
overfit tend to be too 
dependent to the 
development data 
set and the fitted 
relationship falls 
apart when used 
with independent, 
verification data 



Figure 4. Flowcharts depicting the forward selection (top) 
and backward elimination (bottom) methods of choosing 
model predictors (Anderson et al. 2012). 


(Wilks 2006). 

Dr. Huddleston used two meth- 
ods of predictor selection for this 
task; forward selection and backward 
elimination (Figure 4). In many cas- 
es, the two selection methods chose 
different predictors for the final mod- 
el. Predictors were added or re- 
moved based on a p-value of a strin- 
gent 0.01 significance level in order 
to match the level selected by Ma- 
zany et al. (2002). The predictors se- 
lected for final equations for the 2- 
hour and 9-hour forecasts using both 
the forward selection and backward 
elimination methods ranged from 8 to 
19 variables. 


Relative Operating Characteristics 

Dr. Huddleston developed rela- 
tive operating characteristics (ROC) 
curves for each model as a graphical 
way of showing the model’s ability to 
correctly anticipate the occurrence or 
non-occurrence of lightning. The 
ROC curve is useful in helping to 
identify an optimum warning criterion 
by indicating the trade-off between 
misses and false alarms (Mason et 
al. 1999). Generally, for a skillful fore- 
cast system, the ROC curve bends 
towards the top left, where hit rates 
are larger than false alarm rates and 
the area under the curve (AUC) is 
greater than 0.5. If the curve is close 
to the 45° diagonal, the AUC is near 
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^5 and the forecast system does not 
provide any useful information. If the 
curve lies below the 45° diagonal, the 
AUC is less than 0.5 and the forecast 
system provides negative skill 
(Mason and Graham 1999). Mason 
and Graham (1999) provides a sim- 
ple transformation, S, such that 
S = 2 x (AUC - 0.5). The range of S 
is from 1 for a perfect forecast to -1 
for the worst forecast, with 0 indicat- 
ing no skill. 

The ROC curves for the 2-hour 
forecast logistic regression variables 
determined by using the backward 
elimination method are shown in Fig- 
ure 5 along with the ROC curves for 
the Objective Lighting Probability tool 
and the logistic regression equation. 
The logistic regression equation con- 
tains the Objective Lightning Proba- 
bility as a predictor, along with the 
seven changes in GPS-IPW values 
shown in the Figure 5 legend. The 
backward elimination method for the 
2-hour forecast model was one of the 
best performers in terms of AUC and 
had fewer predictors in the final mod- 
el; therefore Dr. Huddleston chose 
this model for illustration. From this 
graphic representation it is easily 
seen that the seven additional GPS- 
IPW predictors provide little addition- 
al information to improve predictabil- 
ity. The specific S values for the ROC 
curves, each predictor, and the AUC 
are shown in Table 2. Clearly, the 
Objective Lightning Probability pre- 
dictor accounts for most of the varia- 
bility in the logistic regression equa- 
tion with the remaining predictors of- 
fering not much more predictability 
than would be expected by random 
chance. 

Reliability Diagram 

A reliability diagram for the lo- 
gistic regression equation for the 2- 
hour forecast backward elimination 
method is shown in Figure 6. In the 
reliability diagram, the dashed diago- 
nal line represents perfect reliability 
and the red curve represents the reli- 
ability of the forecast equation. The 
histogram in the lower right shows 
the frequency of the number of ob- 
servations in each probability range. 
When the curve is below the dashed 

V 


Receiver Operating Characteristics (ROC) Curves for each variable in the 
Logistic Regression Equation with p-value < 0.01 
2 hour lead time 
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Figure 5. ROC curve for each variable in the 2-hour forecast regression equation 
using the backward elimination method of variable selection. 


Table 2. Area under the ROC curve and forecast skill in- 
dex, S, for each variable in the 2-hour forecast logistic 
regression equation using the backward elimination 
method of variable selection. 

Variable 

Area Under 
Curve (AUC) 

S 

Objective Lightning Probability 

0.781 

0.56 

A GPS-IPW (current - 3.0 hr) 

0.654 

0.31 

A GPS-IPW (current -5.5 hr) 

0.668 

0.34 

A GPS-IPW (current - 18.0 hr) 

0.565 

0.13 

A GPS-IPW (current - 18.5 hr) 

0.565 

0.13 

A GPS-IPW (current - 20.0 hr) 

0.571 

0.14 

A GPS-IPW (current -21.0 hr) 

0.589 

0.18 

A GPS-IPW (current - 24.0 hr) 

0.627 

0.25 

Logistic Regression Probability 

0.828 

0.66 


line, the equa- 
tion over- 
forecasted 
lightning oc- 
currence. 

When the 
curve is 
above the 
dashed line, 
the equation 
under- 
forecasted 
lightning oc- 
currence. At 
values of 0.2 
and below, 
the logistic 
regression 
equation performed well, but above 
0.2 the reliability curve of the equa- 
tion is below the dashed line indicat- 
ing that it over-forecasted lightning 
occurrence. Regardless of the selec- 
tion method used, the logistic regres- 
sion equations only produced light- 
ning indexes greater than 0.4 only 
0.1% of the time. Therefore these 
values were treated as extreme and 
not shown in the reliability diagram. 


by a linear combination of other inde- 
pendent variables in the model. Multi- 
collinearity can make the predictor 
variables redundant in some cases. 
Of the 50 candidate predictor varia- 
bles, 48 are a linear combination of 
the current GPS-IPW value and one 
of the half hour interval values be- 
tween 0.5 and 24 hours. Therefore 
there could be some multicollinearity 
of the independent variables. 


Predictor Collinearity 

Multicollinearity occurs when two 
or more independent variables in the 
model are approximately determined 


It is common to evaluate a scree 
plot to determine how many predic- 
tors should be included in a model. 
The scree plot is a graphical repre- 
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Reliability Diagram for 2 Hour Lead Time 
Logistic Regression Equation 
Using Backward Elimination Method 
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Figure 6. Reliability diagram for the 2-hour forecast equation using the backward 
elimination method of variable selection. The dashed diagonal line represents 
perfect reliability and the red curve represents the reliability of the 2-hour forecast 
equation. The histogram in the lower right shows the frequency of the number of 
observations in each probability range. The x-axis of the histogram is the same as 
the forecast probability axis on the reliability diagram. 


sentation of the incremental variance 
accounted for by each predictor in 
the model. Generally, the number of 
predictors that should be in the mod- 
el is limited to those with an eigenval- 
ue > 1*. A predictor with an eigenval- 
ue of < 1 means the variable is not 
contributing an average amount to 
explaining the variance in the model 
(Walker and Maddan 2009). Based 
on this analysis and the results of the 
above ROC diagram, Dr. Huddleston 
developed new equations so she 
could examine the differences in the 
performance metrics. A scree plot for 
the 2-hour forecast equation for the 
backward elimination method of pre- 
dictor selection is shown in Figure 7. 

Dr. Huddleston examined the ei- 
genvalues for the 2-hour and 9-hour 

*ln factor analysis, a component's eigenvalue 
is the amount of variance the component ex- 
plains. The major reason for this is the eigen- 
value's definition as a weighted sum of 
squared correlations. However, the actual vari- 
ance of the component scores also equals the 
eigenvalue. Thus in factor analysis the "factor 
variance" and "amount of variance the factor 
explains" are equal. Therefore the two phrases 
are often used interchangeably, even though 
conceptually they stand for very different 
quantities (Darlington 2012). 


forecast models using both predictor 
selection methods to further reduce 
the number of predictors in the equa- 
tions. The number of predictors was 
reduced to two in each model. One of 
the predictors that remained in all 
models was the Objective Lightning 
Probability tool output. 


Equation Performance 

The performance metrics for the 
equations perform differently depend- 
ing on the lightning threshold index 
value chosen. The accuracy 
measures of probability of detection 
(POD), hit rate (HR), false alarm rate 
(FAR), Kuipers skill score (KSS) and 
Operational Utility Index (OUI) were 
chosen to match the performance 
measures used in a previous GPS- 
IPW study performed at KSC by 
Kehrer et al. (2008). The OUI is a 
locally developed performance metric 
used to emphasize personnel safety 
(Kehrer et al. 2008). The equation for 
OUI is [(3 x POD) + (2 x KSS) - (1 x 
FAR)]/6. Performance metrics for 
each model included the HR, FAR, 
POD, KSS, OUI, and bias at various 
lightning index thresholds. The light- 
ning index threshold is the point at 
which lightning is predicted when the 
model probability falls above the 
threshold and not predicted when 
model probability falls below the 
threshold (Kehrer et al. 2008). To 
maximize the OUI, Dr. Huddleston 
had to reduce the lightning threshold 
index to 0.05, but even then the OUI 
was not as good as the model by 
Kehrer et al. (2008) for either the 2- 
hour or 9-hour forecast model. Per- 
formance metrics for the 2-hour fore- 
cast using the forward, backward, 
and two-predictor forward and back- 
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Figure 7. Scree plot for each predictor in the 2-hour forecast regression equation 
using the backward elimination method of predictor selection. 
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''^vvard selection methods and light- 
ning index values are shown in Ta- 
ble 3. Performance metrics for the 
9-hour forecast were similar and 
will be shown in the final report. 

Conclusions 

Dr. Huddleston investigated the 
utility of using GPS-IPW and output 
from the AMU Objective Lightning 
Probability tool to predict the proba- 
bility of lightning occurrence at the 
half-hour resolution of the GPS- 
IPW output. Using the proven 
methodology of multiple logistic re- 
gression to evaluate the binary pre- 
dictand, she evaluated a total of 50 
candidate predictors to determine a 
subset of predictors that affected 
the predictand. The forward selec- 
tion method and the backwards 
elimination method were used to 
select the predictor. 

Although previous studies 
showed the GPS-IPW values to be 
promising in forecasting lightning, 
the results of this study did not find 
them to be very useful. This is likely 
because the level of noise in the 
Objective Lightning Probability tool, 
which dominates the regression 
equations, is greater than the in- 
crease in predictive capability of- 
fered by the inclusion of the GPS- 
IPW data. The Objective Lightning 
Probability tool was designed to 
predict the probability of lightning 
for the day between the hours of 
0700 to 2400 local time. The equa- 
tions used in the Objective Light- 
ning Probability tool were not de- 
signed for the temporal resolution 
of one half hour. As a result, this 
study demonstrated that inclusion 
of the GPS-IPW data into the AMU 
objective lightning probability as a 
predictor in the equations did not 
improve model performance. 

Final Report 

Dr. Huddleston began writing 
the final report. 

For more information contact 
Dr. Lisa Huddleston at 321-853-8217 

or nsa.l.huddleston@nasa.gov . 


— 1\ 


Table 3. Comparison of accuracy measurements, skill scores, and bias for 
the 2-hour forecast equations, using the backward elimination method, for- 
ward selection method, and the truncated forecast equations using the 
backward elimination method and forward selection method. The scores are 
shown for a range of lightning index threshold values. 

Index 

Selection 

Method 

Hit 

(%) 

POD 

(%) 

FAR 

(%) 

KSS 

(%) 

OUI 

(%) 

Bias 

0.00 

Backward 

4.2 

100 

95.8 

0.0 

34.0 

32.91 

Forward 

4.2 

100 

95.8 

0.0 

34.0 

23.91 

2 predictor back 

4.2 

100 

95.8 

0.0 

34.0 

23.91 

2 predictor fwd 

4.2 

100 

95.8 

0.0 

34.0 

23.91 

0.05 

Backward 

76.9 

72.4 

87.9 

49.5 

38.1 

5.98 

Forward 

76.1 

76.7 

87.7 

52.8 

41.3 

6.24 

2 predictor back 

74.9 

74.7 

88.5 

49.7 

39.2 

6.49 

2 predictor fwd 

73.9 

78.4 

88.5 

52.1 

41.8 

6.81 

0.1 

Backward 

87.8 

42.9 

84.5 

32.7 

18.3 

2.77 

Forward 

87.2 

44.0 

85.1 

33.1 

18.9 

2.95 

2 predictor back 

87.1 

43.2 

85.4 

32.2 

18.1 

2.95 

2 predictor fwd 

86.6 

46.3 

85.2 

34.7 

20.5 

3.12 

0.2 

Backward 

94.2 

12.8 

79.9 

10.6 

-3.4 

0.64 

Forward 

94.5 

11.1 

79.4 

9.2 

-4.6 

0.54 

2 predictor back 

94.6 

7.7 

83.0 

6.0 

-8.0 

0.45 

2 predictor fwd 

94.3 

11.9 

80.0 

9.9 

-4.1 

0.60 

0.3 

Backward 

95.4 

3.1 

81.7 

2.5 

-11.2 

0.17 

Forward 

95.5 

3.4 

74.5 

3.0 

-9.7 

0.13 

2 predictor back 

95.6 

1.1 

85.7 

0.8 

-13.4 

0.08 

2 predictor fwd 

95.5 

2.8 

76.8 

2.4 

-10.6 

0.12 

0.32 

Backward 

95.4 

1.7 

86.7 

1.2 

-13.2 

0.13 

Forward 

95.6 

2.6 

76.9 

2.2 

-10.8 

0.11 

2 predictor back 

95.6 

0.9 

85.7 

0.6 

-13.7 

0.06 

2 predictor fwd 

95.6 

2.6 

76.3 

2.2 

-10.7 

0.11 

0.4 

Backward 

95.6 

0.3 

94.1 

0.1 

-15.5 

0.05 

Forward 

95.7 

1.1 

81.0 

0.9 

-12.6 

0.06 

2 predictor back 

95.7 

0.6 

84.6 

0.4 

-13.7 

0.04 

2 predictor fwd 

95.7 

0.3 

91.7 

0.2 

-15.1 

0.03 

0.5 

Backward 

95.7 

0.0 

N/A 

-0.1 

N/A 

0.03 

Forward 

95.7 

0.0 

N/A 

-0.1 

N/A 

0.02 

2 predictor back 

95.8 

0.0 

100.0 

-0.1 

-16.7 

0.01 

2 predictor fwd 

95.8 

0.0 

100.0 

-0.06 

-16.7 

0.01 
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MESOSCALE MODELING 


Range-Specific High- 
Resolution Mesoscale 
Model Setup 
(Dr. Watson) 

The ER and WFF would benefit 
greatly from high-resolution 
mesoscale model output to better 
forecast a variety of unique weather 
phenomena. Global and national 
scale models cannot properly resolve 
important local-scale weather fea- 
tures at each location due to their 
horizontal resolutions being much too 
coarse. A properly tuned model at a 
high resolution would provide that 
capability. This is the first phase in a 
multi-phase study in which the WRF 
model will be tuned individually for 
each range. The goal of this phase is 
to tune the WRF model based on the 
best model resolution and run time 
while using reasonable computing 
capabilities. The ER and WFF sup- 
ported tasking the AMU to perform a 
number of sensitivity tests in order to 
determine the best model configura- 
tion for operational use at each of the 
ranges. 

Model Test Cases for WFF 

While Dr. Watson was on mater- 
nity leave, Dr. Bauman continued to 
run model test cases for WFF using 
data from 1-30 April 2012 and 1-14 
November 201 1 . The model configu- 
rations are: 

• Configuration 1: Advanced Re- 
search WRF (ARW) core, 2 km 
outer domain and 0.67 km inner 
domain, Lin microphysics 
scheme, Yonsei University PBL 
scheme (Lin-Yonsei), 

• Configuration 2: ARW core, 2 km 
outer domain and 0.67 inner do- 
main, Ferrier microphysics 
scheme, Yonsei University PBL 
scheme (Ferrier-Yonsei), and 

• Configuration 3: ARW core, 2 km 
outer domain and 0.67 inner do- 
main, WDM6 microphysics 


scheme, Yonsei University PBL 
scheme (WDM6-Yonsei). 

• Configuration 4: Non-hydrostatic 
Mesoscale Model (NMM) core, 3 
km outer domain and 1 km inner 
domain, Ferrier microphysics 
scheme, Mellor-Yamada-Janjic 
(MYJ) planetary boundary layer 
(PBL) scheme (NMM 3/1). 

Ms. Crawford and Ms. Shafer 
continued to process the model out- 
put generated by Dr. Bauman. They 
calculated verification statistics for 
the April model runs using hourly sur- 
face and sounding observations. 

On her return from maternity 
leave, Dr. Watson finished running 
four of the NMM forecasts for 6-7 No- 
vember 201 1 and then calculated the 
verification statistics. When examin- 
ing the model output from April 2012, 
Dr. Watson noticed that an inadvert- 
ent change had been made to a run- 
time parameter in some of the com- 
pleted runs. She reran 30 model fore- 
casts and recomputed the verification 
statistics. Dr. Watson also noticed 
erroneous data in the ME- 
TAR observations for both 
April and November. At 
each of the sites, there 
were many instances in 
which both the wind direc- 
tion and wind speed were 
recorded as 0. The data 
looked suspect to Dr. Wat- 
son, therefore she removed 
them from the observation- 
al dataset. 

Test Case Verification 

Dr. Watson recomputed 
the verification statistics for 
the temperature and wind 
data at each METAR site 
once the erroneous data 
were removed. The recal- 
culated Mean Error (ME) 
and Root Mean Square Er- 
ror (RMSE) data were then 
used to repopulate the col- 


or-coded Good-Bad-Neutral (GBN) 
tables that Ms. Crawford and Ms. 
Shafer created. The GBN tables 
show which model configuration per- 
formed the best and worst. A ‘good’ 
rating indicates that the model config- 
uration had the lowest ME or RMSE 
values among the different configura- 
tions. A ‘bad’ rating had the highest 
ME or RMSE values and ‘neutral’ fell 
in the middle. 

Table 4 contains GBN values for 
the combined 30-day (April) ME and 
RMSE of the three ARW configura- 
tions at each of the seven METAR 
and four buoy sites. Table 5 contains 
the combined seven-day (1-7 April) 
ME and RMSE GBN values for the 
three ARW and one NMM configura- 
tion at three of the METAR sites. The 
NMM was only run for 1-7 April and, 
therefore, not compared to the 30- 
day results for the three ARW config- 
urations. Results indicate that for 1- 
30 April, Configuration 1 (Lin-Yonsei) 
outperformed the other configura- 
tions, while Configuration 4 (NMM) 
performs the best for 1-7 April. 


Table 4. GBN totals from the ME and RMSE 
values for the three ARW configuration 
forecasts at seven METAR and four buoy sites 
during 1-30 April 2012 in the WFF vicinity. 

1-30 April 2012 Totals 



Good 

Neutral 

Bad 

Configuration 1 

45 

13 

22 

Configuration 2 

27 

29 

23 

Configuration 3 

9 

35 

36 


Table 5. GBN totals from the ME and RMSE 
values for the three ARW and one NMM 
configuration forecasts at three METAR sites 
from 1-7 April 2012 in the WFF vicinity. 

1-7 April 2011 Totals 


Configuration 1 

Good 

3 

Neutral 

15 

Bad 

6 

Configuration 2 

6 

11 

7 

Configuration 3 

0 

18 

6 

Configuration 4 

15 

3 

5 
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The same GBN values are 
shown for 1-14 and 1-7 November in 
Tables 6 and 7, respectively. Again, 
the NMM was only run for 1-7 No- 
vember and, therefore, not compared 
to the 14-day results for the three 
ARW configurations. Configuration 2 
(Ferrier-Yonsei) performs the best for 
1-14 November, while the Configura- 
tion 4 (NMM) performs the best for 
the 1-7 November period. It should 
be noted that differences in ME and 
RMSE for all configurations were 
small. A detailed breakdown of the 
statistical differences will be provided 
in the final report. 

Dr. Watson compared precipita- 
tion forecasts from all model runs. 
She compared the 24-hour forecast 
accumulated rainfall to the 24-hour 
accumulation of observed rain using 
the National Centers for Environmen- 


tal Prediction (NCEP) 
Stage-1 1 analysis data. Re- 
sults (not shown) indicate 
that Configuration 2 
(Ferrier-Yonsei) produced 
the best precipitation fore- 
cast. 

For more information 
contact Dr. Watson at wat- 
son.leela@ensco.com or 
321-853-8264. 


Table 6. GBN totals from the ME and RMSE 
values for the three ARW configuration 
forecasts at seven METAR and four buoy sites 
during 1-14 November 2011 in the WFF vicinity. 

1-14 November 2011 Totals 



Good 

Neutral 

Bad 

Configuration 1 

18 

13 

25 

Configuration 2 

26 

15 

15 

Configuration 3 

12 

28 

16 


Table 7. GBN totals from the ME and RMSE 
values for the three ARW and one NMM 
configuration forecasts at three METAR sites 
from 1-7 November 2011 in the WFF vicinity. 

1-7 November 2011 Totals 


Configuration 1 

Good 

4 

Neutral 

14 

Bad 

6 

Configuration 2 

0 

20 

4 

Configuration 3 

6 

10 

7 

Configuration 4 

15 

1 
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AMU ACTIVITIES 


AMU Operations 

AMU Tasking 

In preparation for the upcoming 
AMU Tasking meeting, AMU team 
members contacted and met with 
customers to determine possible 
tasks for the next year. The custom- 
ers and the AMU submitted tasks to 
Dr. Huddleston. After receiving all 
proposals, the AMU staff prepared 
responses including the estimated 
time to complete each task and the 
proposed deliverables. Dr. Huddle- 
ston scheduled the tasking meeting 
on 16 November. 

Visitors 

The AMU team attended a meet- 
ing with the KSC Weather Office, the 
and Dr. Luciana Pires of the Depart- 
ment of Science and Aerospace 
Technology in Sao Jose dos Cam- 
pos, Brazil. Dr. Pires was interested 
in collaborating with NASA for her 
post-doctoral research at the Univer- 
sity of Georgia in the area of atmos- 
pheric modeling. The group provided 
with several ideas for her to pursue. 

Mr. Paul O’Brien, the new Light- 
ning Advisory Panel (LAP) member, 

V 


and Mr. Joe Mazur of Aerospace 
Corporation visited the AMU to tour 
the lab and discuss technology tran- 
sition of weather research results to 
operations. 

The AMU team attended the 
presentation given by the Plymouth 
State University summer interns Colt 
Sholton and Kevin Lupo and their 
advisor, Dr. Jim Koermer. Their work 
was a continuation research being 
done at the AMU over the last few 
summers to improve 45 WS convec- 
tive wind forecasts. 

Dr. Bauman presented an over- 
view briefing of the AMU to the new 
AMU co-liaisons from the 45 WS: 

1 Lt Schubeck and Mr. Craft. They 
were particularly interested in the 
AMU tasking process. All AMU team 
members attended. 

Conferences and Training 

Dr. Bauman and Ms. Shafer 
each prepared and submitted an ab- 
stract to the 2013 American Meteor- 
ological Society Meeting in Austin, 
Texas. Both were accepted for oral 
presentation. Dr. Bauman and co- 
author Mr. Flinn of the 45 WS sub- 
mitted an abstract titled “Integrating 


Wind Profiling Radars and Radio- 
sonde Observations with Model 
Point Data to Develop a Decision 
Support Tool to Assess Upper-Level 
Winds for Space Launch”. Ms. Shaf- 
er and co-author Mr. Brock of the 30 
OSS submitted an abstract titled 
“Determining the Probability of Vio- 
lating Upper-Level Wind Constraints 
for the Launch of Minuteman III Bal- 
listic Missiles at Vandenberg Air 
Force Base”. Ms. Shafer is also a co 
-author with four members of the 45 
WS on an abstract titled 
“Communicating the Threat of a 
Tropical Cyclone to the Eastern 
Range”. That abstract was submit- 
ted by the 45 WS. 

Dr. Huddleston and the AMU 
staff planned, set up and manned 
the AMU booth at the KSC Innova- 
tion Expo on 6 September. They met 
people from other groups on KSC 
who indicated the AMU may be able 
to help them with some of their pro- 
jects that depend on weather. 

All AMU team members attend- 
ed the Day Of Launch Working 
Group technical interchange meet- 
ing at KSC on 19-20 September. 
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Security 

Dr. Bauman and Ms. Shafer sub- 
mitted the monthly Asset Import Ta- 
ble spreadsheet to KSC IT Security. 
This is a new recurring IT Security 
requirement. 

Equipment and Software 

Mr. Magnuson from ENSCO’s 
Aerospace Sciences and Engineer- 
ing Division upgraded the AMU and 
Range Weather Operations Ad- 
vanced Weather Information Pro- 
cessing System (AWIPS). The up- 
grade included the high resolution 
Rapid Refresh model used to sup- 
port AMU-developed tools 

Dr. Bauman, Dr. Watson and Mr. 
Magnuson surveyed potential loca- 
tions for the two new AMU modeling 
clusters with Mr. Brown and Ms. Wil- 
son of NASA KSC. They selected a 
server room located in the KSC 
Emergency Operations Center in the 
Launch Control Center building. 

Dr. Watson and Dr. Bauman had 
a telecon with Mr. Case and Mr. Za- 
vodsky of the Short-term Prediction 
Research and Transition Center 
(SPoRT) to discuss using SPoRT 
datasets in future AMU modeling 
work. Mr. Case provided information 
and articles about several datasets 
that can be used to initialize the 
WRF model for better results. Mr. 
Zavodsky provided information on 
his work with data assimilation that 
will assist the AMU in the next phase 
of the Range Modeling task. Through 
the course of the telecon, the AMU 
learned of a NASA-proprietary WRF 


model (NU-WRF) developed at 
SPoRT that may be of use in future 
AMU modeling tasks. With Mr. 

Case’s help, Dr. Watson got in touch 
with the NASA/NU-WRF representa- 
tive and obtained permission to use 
the software. 

Dr. Bauman submitted a paper 
titled “A New Technique to Display 
and Assess Upper-Level Wind Fore- 
casts on Day-of-Launch using Com- 
mercial Off-the-Shelf Software” to the 
2012 Research and Technology 
(R&T) Annual Report for Kennedy 
Space Center's technology projects. 
The report, with its wide distribution, 
provides NASA, academia and in- 
dustry with a broad overview of 
KSC's accomplishments in technolo- 
gy development. 

Dr. Bauman entered an abstract 
in the 2012 Best of KSC Software 
Award competition for the Assessing 
Upper-level Winds on Day-of-Launch 
task software. By entering this com- 
petition, Dr. Bauman was required to 
submit a New Technology Report 
which is titled “Development of a De- 
cision Support Tool to Assess Upper- 
Level Winds for Space Launch". 

AMU Chiefs Technical 

Activities 

(Dr. Huddleston) 

Dr. Huddleston updated VBA 
code to automatically collect model 
output statistics in an Excel spread- 
sheet for Mr. McAleenan of the 45 
WS. They use this spreadsheet to 
calculate skill scores of the GFS 


model for predicting precipitation and 
lightning from Day-1 to Day-7 and to 
compare these skill scores to those 
from forecasts made by 45 WS oper- 
ational personnel. 

Dr. Huddleston attended the 
Lightning Advisory Panel meeting on 
July 25-26. 

Dr. Huddleston conducted some 
calibration runs for the Infrared Ther- 
mometer (IRT) IPW project with Dr. 
Merceret. This project uses inexpen- 
sive commercially available IRTs to 
measure total column water vapor in 
the KSC/CCAFS area as outlined in 
Mims et al. (2011). 

Dr. Huddleston reviewed two pro- 
posals for the NASA Experimental 
Program to Stimulate Competitive 
Research (EPSCoR) program titled 
“Understanding the Atmospheres of 
Hot Earths and the Impact on Solar 
System Formation,” and “Research 
Portfolio for Inaugural Aerospace 
Science Doctoral Program.” 

Dr. Huddleston completed and 
submitted a review of a NASA EP- 
SCoR project at University Of Alaska 
Fairbanks to improve the accuracy of 
wildfire weather models by combin- 
ing meteorological measurements, 
imaging spectrometers, and synthet- 
ic aperture radar with observations of 
burn activity and intensity. 

Dr. Huddleston completed a 
Princeton University online statistics 
course available through Coursera to 
learn the R statistical software pack- 
age for possible use in future AMU 
tasks. 


V. 
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LIS T OF A CRON YMS 


30 SW 30th Space Wing 

30 OSS 30th Operational Support Squadron 
45 RMS 45th Range Management Squadron 
45 OG 45th Operations Group 

45 SW 45th Space Wing 

45 SW/SE 45th Space Wing/Range Safety 
45 WS 45th Weather Squadron 

AFSPC Air Force Space Command 

AFWA Air Force Weather Agency 

AMPS Automated Meteorological Profiling System 

AMU Applied Meteorology Unit 

ARW Advanced Research WRF 

AUC Area Under the Curve 

AWIPS Advanced Weather Information Processing 
System 

CCAFS Cape Canaveral Air Force Station 

CSR Computer Sciences Raytheon 

EDT Eastern Daylight Time 

ER Eastern Range 

ESRL Earth System Research Laboratory 

FAR False Alarm Rate 

FSU Florida State University 

FTP File Transfer Protocol 

GBN Good Bad Neutral 

GFS Global Forecast System 

GPS Global Positioning System 

GUI Graphical User Interface 

HR Hit Rate 

HTTP Hypertext Transfer Protocol 

IPW Integrated Precipitable Water 

IRT Infrared Thermometer 

IT Information Technology 

JSC Johnson Space Center 

KSC Kennedy Space Center 

KSS Kuiper Skill Score 


LAP Lightning Advisory Panel 

LWO Launch Weather Officer 

MCO Orlando International Airport 

ME Mean Error 

MIDDS Meteorological Interactive Data Display Sys- 
tem 

MLB Melbourne International Airport 

MSFC Marshall Space Flight Center 

NAM North American Mesoscale model 

NCEP National Centers for Environmental 

Prediction 

NLDN National Lightning Detection Network 
NMM Non-hydrostatic Mesoscale Model (WRF) 

NOAA National Oceanic and Atmospheric 

Administration 

NWS MLB National Weather Service in Melbourne, Fla. 
OUI Operational Utility Index 

POD Probability of Detection 

POR Period of Record 

RAP Rapid Refresh model 

RMSE Root Mean Square Error 

ROC Relative Operating Characteristics 

RUC Rapid Update Cycle model 

SMC Space and Missile Center 

SPoRT Short-term Prediction Research and Transi- 
tion Center 

SS Brier Skill Score 

URL Uniform Resource Locator 

USAF United States Air Force 

VAFB Vandenberg Air Force Base 

VBA Visual Basic for Applications 

WFF Wallops Flight Facility 

WRF Weather Research and Forecasting Model 

XMR CCAFS 3-letter identifier 
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The AMU has been in operation since September 1991. Tasking is 
determined annually with reviews at least semi-annually. 

AMU Quarterly Reports are available on the Internet at http://science.ksc.nasa.gov/amu/ . 

They are also available in electronic format via email. If you would like to be added to the email distribution list, 
please contact Ms. Winifred Crawford (321-853-8130, crawford.winnie@ensco.com) . 

If your mailing information changes or if you would like to be removed from the distribution list, please notify 
Ms. Crawford or Dr. Lisa Huddleston (321-861-4952, Lisa. L. Huddleston@nasa.gov) . 
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